Stitching processes are known in the art, and comprise combining (or merging) digital fractional images (or videos, i.e, sequences of images), each one covering a fractional angle of vision, to produce digital panoramic images (or videos) covering a wider angle of vision (that is, bigger than the angle of vision covered by a single digital fractional image or video). In order to produce said digital panoramic images (or videos), it is necessary that the angle of vision of each digital fractional image (or video), partially overlaps with the angle of vision of at least another digital fractional image (or video).
Stitching processes are usually used with:                a) photographic digital images, in which case two or more fractional photographic images—with fractional and partially overlapping angles of vision—are merged to produce a single panoramic digital image; and        b) with digital video sequences, in which case two or more video sequences of digital fractional images, with fractional and partially overlapping angles of vision, are merged to produce a panoramic video sequence;        
Digital images are formed by a fixed number of rows and columns of pixels. Pixels are the smallest individual elements (or dots) in an image, holding values that represent the color values (for example, color, brightness and saturation) of a given color at a specific point of said digital image. On the other hand, video sequences are, merely, sequences of digital images, each image comprising the same fixed number of rows and columns of pixels.
Panoramic images (or videos) obtained with said stitching process are (preferably but not necessarily) 360° panoramic images (or videos). In fact, stitching is a common technique used by most 360° camera manufacturers.
The most common stitching processes known in the art, essentially comprise the following steps:                i) obtaining the digital fractional images, which cover fractional and partially overlapping angles of vision; or alternatively obtaining the digital fractional video sequences, which cover fractional and partially overlapping angles of vision;        ii) calibrating the digital fractional images, in order to reduce exposure and chromatic differences between adjacent digital fractional images; or alternatively calibrating the video sequences of digital fractional images, in order to reduce exposure and chromatic differences between digital fractional video sequences;        iii) finding correspondences between adjacent digital fractional images; or alternatively finding correspondences between digital fractional video sequences;        iv) modifying the digital fractional images, in view of the correspondences found in the step above, for improving the alignment between adjacent digital fractional images; or alternatively modifying the digital fractional video sequences, in view of the correspondences found in the step above, for improving the alignment of digital fractional video sequences;        v) aligning adjacent digital fractional images together and making them fit into a predefined compositing surface (or projection), to form a digital panoramic image; or alternatively aligning digital fractional video sequences together and making them fit into a predefined compositing surface (or projection), to form a panoramic video sequence; and        vi) blending the adjacent digital fractional images which form the digital panoramic image in order to minimize seams (i.e., errors or imperfections existing in the overlapping areas of adjacent digital fractional images); or alternatively blending the digital fractional video sequences which form the digital panoramic video sequence in order to minimize seams.        
Some prior art stitching processes contemplate, in the correspondence finding step iii) mentioned above, the use of different already-known “feature detection” algorithms, such as corners algorithms, blobs algorithms, Harris corners algorithms or DoG algorithms (difference-of-Gaussian algorithms).
Some prior art stitching processes contemplate, in the modification step iv) mentioned above, modifications such as pure rotation, pure translation, scaling of the digital fractional images (or alternatively of the sequences of the digital fractional images), and combinations thereof.
Most of prior art stitching processes contemplate, in the aligning step v) mentioned above, selecting a substantial part of each digital fractional image (usually 40% of the rows and columns of pixels forming each fractional image, or more) to form the digital panoramic image (or video).
In order to reduce seams, most of the substantial part of each fractional image/sequence of images usually selected by prior art stitching processes to form the panoramic image/sequence of images, corresponds to the non-overlapping angle of vision thereof.
Some projections employed in the aligning step (step v mentioned above) of stitching processes according to prior art, are rectilinear, cylindrical and spherical.
Some prior art stitching processes contemplate, in the blending step vi) mentioned above, minimizing the intensity difference of overlapping pixels, corresponding to adjacent digital fractional images (or videos).
The above-mentioned stitching processes of the prior art, work properly when all the partial images (or videos) are taken from the same position (called the non-parallax point). In other words, said classic stitching processes are capable of merging all the partial images (videos) seamlessly, even with camera rotations between the shots.
Nevertheless, if the partial images are taken from different positions (i.e., with parallax), errors such as discontinuities could occur when using said classic stitching process.
In fact, references to this technical problem can be found in dedicated forums, and even in the web pages of some of the leading companies of the sector. For example, the following statement could be found in the official forum of the Company VideoStitch Inc.: “Parallax introduces stitching errors, making seams more visible after stitching, usually on objects close to the camera”. “It's impossible to remove it entirely, but there are different ways to reduce if”. More information could be found in the following web site:
«support.video-stitch.com/hc/en-us/community/posts/211167728-What-is-parallax-Howdoes-it-impact-360-video-output»
Further explanations of the stitching errors due to parallax can also be found, for example in the Kolor (commercial name) web site, the stitching software developed by the company GoPro Inc., «www.kolorcom/wiki-en/action/view/Autopano_Video_-_Parallax».
Therefore, there is a need for providing a method capable of correctly merging fractional images (or videos) taken from different positions (with parallax). In particular, in order to obtain stereoscopic images or videos, it is necessary to record fractional images with parallax.
Stereoscopic images (or videos) comprise two images (two sequences of images in the case of videos), one for each eye. The parallax between the two images (between the two sequences of images, in the case of videos) allows the human brain to distinguish depth (i.e., see in 3D). More particularly, nearby objects have greater shift in image position between left and right eyes than far away objects. Consequently, in order to record a stereoscopic image or video, it is not enough to record from a single point, but it is necessary to move the cameras (or to have a multi-camera system) that records this necessary parallax.
Consequently, some stitching processes, intended to form a digital panoramic image (or alternatively, a digital panoramic video sequence) from digital fractional images (or alternatively from digital fractional video sequences) obtained by a rotation of cameras with parallax, have been recently developed in the art.
One example of said recently implemented stitching processes capable of merging fractional images taken with parallax, is the ODS (omni-directional stereo) projection. described in the following web site:
«developers.google.com/vr/jump/rendering-ods-content.pdf»
In order to avoid seam problems associated to the stitching of partial images taken with parallax, said ODS projection selects, during the alignment step, just the central ray of each digital fractional image, to form the digital panoramic image, instead of selecting a substantial part of each partial image/sequence of partial images.
Throughout the present description, it should be understood that a “ray” of a digital image is the set of all digital information corresponding to a particular horizontal direction of said digital image. In other words, a ray of a digital image consists in all the digital information coming from all vertical angles, but corresponding to the same horizontal angle. Consequently, a ray of a digital image is a particular vertical column of said digital image, said vertical column having a width of 50 pixels, or less.
ODS projection is capable of forming digital 360° panoramic stereoscopic images, but it is not capable of forming digital panoramic stereoscopic videos by using just two real (physical) recording video cameras and was not even conceived for that purpose. In fact, it is impossible to use—in practice—the ODS projection with real video cameras, since it would be necessary that a great number of different cameras ((i.e., several hundreds, even thousands) to record video simultaneously. And this unthinkable is in terms of costs and physical space.
In addition, no prior art stitching processes, including ODS algorithm, is capable of forming a seamless digital panoramic video by combining a separate digital fractional video sequence covering a predefined angle of vision with a set of digital fractional images taken at other time, and covering the remaining angle of vision.
The stitching process according to the present invention is intended to address the problems and disadvantages of the prior art, mentioned above.